Multilayer neural networks and Bayes decision theory

There are many applications of multilayer neural networks to pattern classification problems in the engineering field. Recently, it has been shown that Bayes a posteriori probability can be estimated by feedforward neural networks through computer simulation. In this paper, Bayes decision theory is combined with the approximation theory on three-layer neural networks, and the two-category n-dimensional Gaussian classification problem is studied. First, we prove theoretically that three-layer neural networks with at least 2n hidden units have the capability of approximating the a posteriori probability in the two-category classification problem with arbitrary accuracy. Second, we prove that the input-output function of neural networks with at least 2n hidden units tends to the a posteriori probability as Back-Propagation learning proceeds ideally. These results provide a theoretical basis for the study of pattern classification by computer simulation.